Electronic device and control method thereof

ABSTRACT

Disclosed are an electronic device and a control method capable of virtually arranging a product by synthesizing an image of a user&#39;s desired product with a spatial image. The control method of an electronic device, according to the present disclosure, comprises the steps of: capturing a marker object including a handwriting image to obtain an image; obtaining information on at least one product corresponding to the handwriting image; and if one of the at least one product is selected, synthesizing and displaying an image of the product on an area where the marker object image is located, on the basis of the information on the selected product, information on the size of the marker object, and information on the size of the marker object image.

This application is the U.S. national phase of International ApplicationNo. PCT/KR2018/005638 filed May 16, 2018 which designated the U.S. andclaims priority to KR Patent Application No. 10-2017-0103468 filed Aug.16, 2017, the entire contents of each of which are hereby incorporatedby reference.

TECHNICAL FIELD

Apparatuses and methods consistent with the disclosure relate to anelectronic device and a control method thereof, and more particularly,to an electronic device and a control method thereof capable ofsynthesizing an image of a product desired by a user with a space imageto virtually arrange the product.

BACKGROUND ART

In a house, various furniture and electronic devices are arranged.Because furniture or electronic product generally has a large size andis expensive, a consumer determines whether or not to purchase thefurniture or electronic product by considering various factors.Consumers value design as much as practicality when purchasing furnitureand electronic product.

However, it is difficult for the consumer to determine whether or notthe furniture or electronic product matches interior decoration ofhis/her house until the consumer purchases the furniture or electronicproduct and arranges the purchased furniture or electronic product inhis/her house. Therefore, a method of virtually arranging furniture toassist in a purchase of the consumer has been discussed.

However, as a method of arranging virtual furniture according to therelated art, a method of overlapping a furniture image with an imageobtained by capturing an interior in which the consumer desires toarrange the furniture has been mainly used. In case of the methodaccording to the related art, because the arranged virtual furnituredoes not match a size and perspective of an interior image, a state inwhich actual furniture is arranged is not accurately expressed in manycases. Further, even in case that the user may adjust the size andperspective, attributes of the virtual furniture, such as a size andperspective, need to be directly adjusted by the user, and thus it isdifficult to accurately synthesize products.

Meanwhile, as the related art, a technology capable of recognizing amarker and arranging furniture corresponding to the marker exists.However, according to the related art, in case of arranging virtualfurniture having a color or shape desired by the user in a manner inwhich a virtual furniture image is arranged by using a predeterminedmarker, a marker for furniture desired by the user needs to be exist,which is inconvenience.

SUMMARY

The disclosure provides an electronic device and a control methodthereof capable of virtually arranging a desired product based oninformation on a size and a perspective of an image with a simpleoperation by a user, and a method thereof.

According to an embodiment of the disclosure, a control method of anelectronic device includes: capturing a marker object including ahandwriting image to obtain an image; obtaining information on one ormore products corresponding to the handwriting image; and once one ofthe one or more products is selected, synthesizing and displaying animage of the product in and on a region in which the marker object imageis positioned based on information on the selected product, sizeinformation of the marker object, and size information of the markerobject image.

In the displaying, the image of the selected product may be processed byanalyzing size information of the selected product, the size informationof the marker object, the size information of the marker object image,and perspective information of the marker object image.

The handwriting image may include color information, and in theobtaining of the information on one or more products, information on oneor more products corresponding to the handwriting image and the colorinformation of the handwriting image may be obtained.

The control method may further include: once a user command to deleteone of a plurality of product images included in the image is input,deleting the one product image; and performing image synthesis on aregion from which the product image is deleted based on an image patternof a surrounding region of the deleted product image and displaying asynthesized image.

The control method may further include: once a first product image isselected among a plurality of product images included in the image,obtaining first product information corresponding to the first productimage; obtaining one or more other product information corresponding tothe first product information; and once second product information isselected among the one or more other product information, deleting thefirst product image and synthesizing and displaying a second productimage corresponding to the second product information in and on a regionin which the deleted first product image is positioned.

The control method may further include: once one of a plurality ofproduct images included in the image is selected, obtaining productinformation of the selected product image; and recommending new productinformation based on the obtained product information.

The control method may further include: obtaining color information ofthe image; and recommending new product information based on a pluralityof product information included in the image and the color information.

In the displaying, once a user command specifying one of a plurality ofproduct images included in the image is input, the one product imagespecified in the user command may be repositioned and displayed based onthe user command.

The product information may be one of size information, categoryinformation, or price information of a product.

According to another embodiment of the disclosure, an electronic deviceincludes: an image capturer configured to capture an image; a displayconfigured to display the image; an inputter configured to receive auser command; at least one processor; and a memory configured to storeone or more computer programs executed by the at least one processor,wherein the one or more computer programs include: a command to controlthe image capturer to capture a marker object including a handwritingimage to obtain an image; a command to obtain information on one or moreproducts corresponding to the handwriting image; a command to receive aninput selecting one of the one or more products through the inputter; acommand to synthesize and process an image of the selected product in aregion in which the marker object image is positioned based oninformation on the selected product, size information of the markerobject, and size information of the captured marker object image, inresponse to the input; and a command to control the display to controlthe synthesized image.

The command to process the image of the selected product may be acommand to synthesize and process the image of the selected product byanalyzing size information of the selected product, the size informationof the marker object, the size information of the marker object image,and perspective information of the marker object image.

The handwriting image may include color information, and the command toobtain the information on one or more products may be a command toobtain information on one or more products corresponding to thehandwriting image and the color information of the handwriting image.

The one or more computer programs may further include: a command toreceive a user command to delete one of a plurality of product imagesincluded in the image, a command to delete the one product image; and acommand to perform image synthesis and processing on a region from whichthe product image is deleted based on an image pattern of a surroundingregion of the deleted product image.

The one or more computer programs may further include: a command toobtain first product information corresponding to a first product imageonce the first product image is selected among a plurality of productimages included in the image; a command to obtain one or more otherproduct information corresponding to the first product information; anda command to delete the first product image and synthesize and process asecond product image corresponding to second product information in andon a region in which the deleted product image is positioned once thesecond product information is selected among the one or more otherproduct information.

The one or more computer programs may further include: a command toobtain, once one of a plurality of product images included in the imageis selected, product information of the selected product image; and acommand to recommend new product information based on the obtainedproduct information.

The one or more computer programs may further include: a command toobtain color information of the image; and a command to recommend newproduct information based on a plurality of product information includedin the image and the color information.

The one or more computer programs may further include: a command toreposition and process, once a user command specifying one of aplurality of product images included in the image is input, the oneproduct image specified in the user command based on the user command.

The product information may be one of size information, categoryinformation, or price information of a product.

According to another embodiment of the disclosure, a computer-readablerecording medium including a program for controlling an electronicdevice is provided. A control method of an electronic device includes:capturing a marker object including a handwriting image to obtain animage; obtaining information on one or more products corresponding tothe handwriting image; and once one of the one or more products isselected, synthesizing and displaying an image of the product in and ona region in which a marker object image is positioned based oninformation on the selected product, size information of the markerobject, and size information of the marker object image.

As described above, according to various embodiments of the disclosure,it is possible to synthesize a product image having an accurate sizewith a space image, and recommend a product that matches the spaceimage.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a schematic configuration of anelectronic device 100 according to an embodiment of the disclosure.

FIG. 2 is a block diagram illustrating a detailed configuration of theelectronic device 100 according to an embodiment of the disclosure.

FIGS. 3A, 3B, 3C, 4A, 4B, and 4C are illustrative views for describing amethod of arranging a product image by using a marker object accordingto an embodiment of the disclosure.

FIGS. 5A and 5B are illustrative views for describing a method ofchanging a product image included in an image with another product imageaccording to an embodiment of the disclosure.

FIGS. 6A and 6B are illustrative views for describing a method ofdeleting a product image included in a space image according to anembodiment of the disclosure.

FIGS. 7A and 7B are illustrative views for describing a method ofrecommending a product according to an embodiment of the disclosure.

FIGS. 8A and 8B are illustrative views for describing another method ofsynthesizing a product image according to an embodiment of thedisclosure.

FIGS. 9A, 9B, and 9C are illustrative views for describing a method ofrepositioning a specific product image in a space image to anotherregion according to an embodiment of the disclosure.

FIGS. 10A, 10B, 10C, 11A, 11B, 11C, and 11D are illustrative views fordescribing a method of synthesizing a virtual product image according toan embodiment of the disclosure.

FIG. 12 is a flowchart for describing a method for synthesizing avirtual product according to an embodiment of the disclosure.

FIG. 13 is an illustrative view for describing a method of determining asize of a synthesized product image according to an embodiment of thedisclosure.

FIGS. 14A and 14B are illustrative views for describing a method ofsynthesizing a product image by analyzing a perspective of a space imageaccording to an embodiment of the disclosure.

FIGS. 15A, 15B, and 15C are illustrative views for describing the caseof zooming in or out a synthesized space image according to anembodiment of the disclosure.

FIG. 16 is a flowchart for describing the embodiment of FIGS. 13 to 15according to an embodiment of the disclosure.

FIGS. 17A, 17B, 17C, 17D, and 18 are illustrative views for describing amethod of recommending an arrangeable product by analyzing a space imageaccording to an embodiment of the disclosure.

FIG. 19 is a flowchart for describing a process of synthesizing aproduct image according to an embodiment of the disclosure.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

The embodiment may be variously modified and have several forms.Therefore, specific embodiments of the disclosure will be illustrated inthe accompanying drawings and be described in detail in thespecification. However, it is to be understood that the disclosure isnot limited to specific embodiments, but includes all modifications,equivalents, and substitutions without departing from the scope andspirit of the disclosure. In case that it is decided that a detaileddescription for the known art related to the disclosure may obscure thegist of the disclosure, the detailed description will be omitted.

Terms “first”, “second”, and the like, may be used to describe variouscomponents, but the components are not to be construed as being limitedby these terms. The terms are used only to distinguish one componentfrom another component.

Terms used in the disclosure are used only to describe specificembodiments rather than limiting the scope of the disclosure. Singularforms are intended to include plural forms unless the context clearlyindicates otherwise. It will be further understood that terms “include”or “formed of” used in the specification specify the presence offeatures, numerals, steps, operations, components, parts, orcombinations thereof mentioned in the specification, but do not precludethe presence or addition of one or more other features, numerals, steps,operations, components, parts, or combinations thereof.

In the embodiments, a “module” or a “-er/or” may perform at least onefunction or operation, and be implemented by hardware or software or beimplemented by a combination of hardware and software. In addition, aplurality of “modules” or a plurality of “-ers/ors” may be integrated inat least one module and be implemented by at least one processor (notillustrated) except for a “module” or an “-er/or” that needs to beimplemented by specific hardware.

Hereinafter, embodiments of the disclosure will be described in detailwith reference to the accompanying drawings so that those skilled in theart to which the disclosure pertains may easily practice the disclosure.However, the disclosure may be modified in various different forms, andis not limited to embodiments described herein. In addition, in thedrawings, portions unrelated to the description will be omitted toobviously describe the disclosure, and similar reference numerals willbe used to describe similar portions throughout the specification.

Meanwhile, before describing the disclosure in detail, terms used in thedisclosure will be defined. First, a handwriting image refers to animage included in a marker object. For example, the handwriting imagemay be a shape drawn by a user. The marker object is a standardizedobject. Here, the standardization refers to a standard size of anobject. For example, the marker object may be a sheet of A4 paper. Amarker object image is a virtual object arranged in a space image. Aproduct refers to an object arranged or to be arranged in an actualspace of the space image, and a product image refers to a virtual objectarranged or to be arranged in the space image.

The marker object and the handwriting image are not limited to theexamples described above, and may have various forms.

Hereinafter, the disclosure will be described in detail with referenceto the accompanying drawings.

FIG. 1 is a block diagram illustrating a schematic configuration of anelectronic device 100 according to an embodiment of the disclosure.Here, the electronic device 100 may include at least one of, forexample, a smartphone, a tablet personal computer (PC), a mobile phone,an image phone, a desktop personal computer (PC), a laptop personalcomputer (PC), or a netbook computer.

As illustrated in FIG. 1, the electronic device 100 includes an imagecapturer 110, a display 120, an inputter 130, a memory 140, and aprocessor 150.

The image capturer 110 is a component capturing an image of an interiorspace or the like for arranging a product. Specifically, the imagecapturer 110 may capture an image of an interior space including amarker object. Here, the marker object may further include a handwritingimage.

The marker object may be used to obtain information on a size and aperspective of a space image. Here, the marker object is a standardizedobject. For example, the marker object may be a sheet of A4 paper with awidth of 210 mm and a length of 297 mm. Because an actual size of themarker object is given, it is possible to obtain a size and aperspective of the space image based on a size of a marker object imageincluded in the space image, and determine a size and a perspective of aproduct image to be newly synthesized.

The display 120 may display various images. Specifically, the display120 may display an image in which the space image and the product image,captured by the image capturer 110, are synthesized with each other.

The inputter 130 may receive various user commands. The inputter may beimplemented in various forms such as a touch inputter or a voiceinputter.

The memory 140 may store a program and data for controlling theelectronic device 100. In particular, the memory 140 may store one ormore computer programs executed by the processor 150.

The processor 150 controls a general operation of the electronic device100. Specifically, the processor 150 controls the electronic device 100by executing the one or more computer programs stored in the memory 140.However, it is described in the disclosure that the processor 150controls various components of the electronic device 100, forconvenience of explanation.

First, the processor 150 may obtain a space image captured by the imagecapturer 110. As described above, the space image includes a markerobject, and the marker object may include a handwriting image.

The processor 150 may obtain information on one or more productscorresponding to the handwriting image. Specifically, the processor 150may analyze color information and shape information of the handwritingimage to obtain information on one or more products that are similar tothe color information and the shape information.

Once one of information on the one or more products is selected throughthe inputter 130, the processor 150 may synthesize an image of theselected product in a region in which the marker object is positioned,based on size information of the selected product, size information ofthe marker object, and size information of a marker object image.

However, the above-described embodiment is an example briefly describingthe case that an image of the marker object is captured at the front. Incase that the space image is not captured at the front, the processor150 may additionally determine perspective information (or angleinformation) of the marker object image.

The processor 150 may calculate a size and a perspective of a productimage even in case that the space image is not captured at the front, byusing perspective information of the marker object (and marker objectimage) in addition to the size information of the marker object (andmarker object image).

The processor 150 may arrange product images with various forms in thespace image by using the above-described method. Hereinafter, variousembodiments in which the processor 150 adds, deletes, and changesvarious product images included in the space image will be described.Here, a description of a method of calculating size information andperspective information of the product image will be omitted forconvenience of explanation, but the method of calculating sizeinformation and perspective information of the product image is asdescribed above.

The processor 150 may receive a command to delete one of a plurality ofproduct images included in the space image, through the inputter 130.The processor 150 may delete the product image specified in the deletioncommand according to the received input. Here, the processor 150 maysynthesize a region from which the product image is deleted, based on animage pattern of a surrounding background region. Here, the productimage deletion command may be a user command input by long pressing theproduct image, but is not limited thereto. The product image deletioncommand may be various types of commands input by double tapping,swiping, and the like.

Meanwhile, the processor 150 may change a first product image among theplurality of product images included in the space image with a secondproduct image. Specifically, once the first product image is selectedamong the plurality of product images, the processor 150 may obtainfirst product information corresponding to the first product image. Theprocessor 150 may obtain one or more other product informationcorresponding to the obtained first product information. Once a usercommand for selecting second product information among the obtained oneor more product information is input, the processor 150 may delete thefirst product image and then synthesize and process the second productimage corresponding to the second product information in a region inwhich the first product image is positioned.

Here, the second product image corresponding to the second productinformation may be obtained based on marker object information, markerobject image information, and the second product information, asdescribed above. Meanwhile, in case that a size of the second productimage is smaller than a size of the first product image, a region fromwhich the first product image is deleted may not be completely filledwith the second product image. In this case, the processor 150 mayperform image synthesis on the remaining region based on an imagepattern of a surrounding background, as described above.

Meanwhile, once one of the plurality of product images included in thespace image is selected, the processor 150 may obtain productinformation corresponding to the selected product image. The processor150 may recommend new product information based on the obtained productinformation.

For example, in case that a first product is a 51 inch television (TV),the processor 150 may obtain information on one or more 51 inch TVs thatare different from the first product. Alternatively, in case that thefirst product is a blue rectangular product, the processor 150 mayrecommend one or more blue rectangular products that are different fromthe first product.

Meanwhile, once a user command specifying one of the plurality ofproduct images included in the space image is input, the processor 150may reposition the one product image specified in the user command (forexample, the user command input by long pressing the product image).Here, the repositioning of the product includes various behaviors suchas turning or reversing the product, in addition to changing a region inwhich the product is positioned.

FIG. 2 is a block diagram illustrating a detailed configuration of theelectronic device 100 according to an embodiment of the disclosure. Asillustrated in FIG. 2, the electronic device 100 may further include acommunication unit 160 and an image processor 170. However, thecomponents illustrated in FIG. 2 are only an embodiment of thedisclosure, and some components may be added or eliminated as necessary.

The image capturer 110 is a component capable of capturing an image asdescribed above. The image capturer 110 may include a lens, a shutter,an aperture, an image sensor, an analog front end (AFE), and a timinggenerator (TG).

Specifically, the lens (not illustrated) is a component through whichlight reflected from a subject is incident, and may include at least oneof a zoom lens or a focus lens. The shutter (not illustrated) adjusts atime during which light comes into the electronic device 100. Anintensity of light accumulated in an exposed pixel of the image sensoris determined depending on a shutter speed. The aperture (notillustrated) is a component adjusting an intensity of light passingthrough the lens and incident into the electronic device 100. The imagesensor (not illustrated) is a component on which an image of the subjectthat passes through the lens is formed.

As described above, the image capturer 110 may capture a regionincluding a marker object including a handwriting image. Further, theimage capturer 110 may obtain a space image including a marker objectimage corresponding to the marker object.

The display 120 may be implemented by various types of display panels.For example, the display panel may be implemented by various displaytechnologies such as a liquid crystal display (LCD) technology, anorganic light emitting diode (OLED) technology, an active-matrix organiclight emitting diode (AM-OLED) technology, a liquid crystal on silicon(LcoS) technology, and a digital light processing (DLP) technology.Further, the display 120 may be a flexible display and may be coupled inat least one of a front surface region, a side surface region, or a backsurface region of the electronic device 100.

The display 120 may display the space image captured by the imagecapturing unit 110, and in case that various product images are edited(for example, synthesized, deleted, or changed) by the processor 150,the display 120 may display the edited image.

The inputter 130 may receive a user command. The inputter 130 may beconfigured in various forms. For example, the inputter 130 may beimplemented by a touch screen by combining with the display 120 and atouch sensor (not illustrated). However, the inputter 130 is not limitedto such a configuration. The inputter 130 may include a button, may beimplemented by an external remote controller or a microphone for speechinput, or may combine with the image capturer 110 for motion input.

Particularly, the inputter 130 may receive a user command for editing aproduct image. Here, the input user command may be assigned to longpressing, double tapping, swiping, or the like.

The memory 140 may store an operating system (O/S) for driving theelectronic device 100. In addition, the memory 140 may store varioussoftware programs or applications for operating the electronic device100 according to various embodiments of the disclosure. The memory 140may store various information such as various kinds of data input, set,or generated during execution of the programs or the applications.

In addition, the memory 140 may include various software modules foroperating the electronic device 100 according to various embodiments ofthe disclosure, and the processor 150 may execute the various softwaremodules stored in the memory 140 to perform an operation of theelectronic device 100 according to various embodiments of thedisclosure.

Further, the memory 140 may store the space image captured by the imagecapturer 110 and various images received from the outside. To this end,the memory 140 may include a semiconductor memory such as a flashmemory, or the like, or a magnetic storing medium such as a hard disk,or the like.

The communication unit 160 may perform communication with an externaldevice. In particular, the communication unit 160 may include variouscommunication chips such as a wireless fidelity (Wi-Fi) chip, aBluetooth chip, a near field communication (NFC) chip, and a wirelesscommunication chip. Here, the Wi-Fi chip, the Bluetooth chip, and theNFC chip perform communication in a local area network (LAN) scheme, aWi-Fi scheme, a Bluetooth scheme, an NFC scheme, respectively. In caseof using the Wi-Fi chip or the Bluetooth chip, various connectioninformation such as a service set identifier (SSID) and a session key isfirst transmitted and received, communication connection is establishedusing the connection information, and various information may then betransmitted and received. The wireless communication chip means a chipperforming communication according to various communication protocolssuch as Institute of Electrical and Electronics Engineers (IEEE),Zigbee, 3^(rd) generation (3G), 3^(rd) generation partnership project(3GPP), and long term evolution (LTE). Particularly, the communicationunit 160 may receive various information from an external device (forexample, a content server providing a product image). For example, thecommunication unit 160 may receive various interior images, productinformation, and product images from the external device, and store thereceived information in the memory 140.

The image processor 170 may process raw image data captured by the imagecapturer 110 to convert the raw image data into YCbCr data. Further, theimage processor 170 may determine an image black level and adjust asensitivity ratio for each color. In addition, the image processor 170may adjust a white balance level and perform gamma correction, colorinterpolation, color compensation, and resolution conversion.

Particularly, the image processor 170 may edit (for example, add,delete, or change) a product image in a space image to create a newimage. It is a matter of course that the image processor 170 may also beimplemented by a computer program stored in the memory 140 and at leastone processor executing the stored computer program.

Further, the electronic device 100 may additionally include variouscomponents such as various types of sensors (for example, anacceleration sensor, a gyro sensor, and an illuminance sensor), anoutputter (for example, an audio outputter), and a vibrator.

The processor 150 controls a general operation of the electronic device100.

The processor 150 may include a random access memory (RAM), a read onlymemory (ROM), a main central processing unit (CPU), first to n-thinterfaces, and a bus. Here, the RAM, the ROM, the main CPU, the firstto n-th interfaces, and the like may be connected to one another throughthe bus.

An instruction set for booting a system, or the like, is stored in theROM. Once a turn-on command is input to supply power, the main CPU maycopy the O/S stored in the memory 140 to the RAM according to aninstruction stored in the ROM, and execute the O/S to boot the system.Once the booting is completed, the main CPU copies various applicationprograms stored in the memory 140 to the RAM, and executes theapplication programs copied to the RAM to perform various operations.

The main CPU accesses the memory 140 to perform booting using the O/Sstored in the memory 140. In addition, the main CPU performs variousoperations using various programs, contents, data, and the like, storedin the memory 140.

The first to n-th interfaces are connected to the various componentsdescribed above. One of the interfaces may be a network interfaceconnected to an external device through a network.

Hereinafter, a method of arranging a product image according to thedisclosure will be described in detail with reference to FIGS. 3a to 12.

FIGS. 3a to 4c are views for describing a method of arranging a productimage by using a marker object according to an embodiment of thedisclosure.

As illustrated in FIG. 3a , the user may draw, on a marker object (forexample, a sheet of A4 paper), a shape (handwriting image 310) of aproduct that the user desires to arrange. Here, the handwriting image310 includes shape information and color information. For example, inFIG. 3a , the handwriting image may be an image having a yellowhorizontally long rectangular shape. As illustrated in FIG. 3b , theuser may position a marker object in a specific region (for example, awall) of a space (for example, a living room) in which a product is tobe arranged. The electronic device 100 may capture the space includingthe marker object to obtain an image of the space.

The electronic device 100 may synthesize a product image with a spaceimage. Specifically, the electronic device may obtain one or moreproduct information corresponding to the handwriting image by analyzingthe handwriting image 310. Here, the one or more product information maybe product information stored in the memory 140 or product informationstored in an external server. The product information may be obtained byusing information on a position where the marker object is attached, inaddition to the handwriting image 310. For example, in case of FIG. 3c ,the marker object is attached to the wall, and thus the electronicdevice 100 may obtain information on a product that may be arranged on awall.

The electronic device 100 may display one or more product information.In case that one of the one or more product information is selected, theelectronic device 100 may generate a product image corresponding to theselected product information. As described above, the product image maybe obtained based on actual size information of the marker object, sizeinformation and perspective information of the marker object imageincluded in the space image, and actual size information of the selectedproduct.

Specifically, the electronic device 100 may calculate a size of theproduct image with respect to the size of the product based on a ratioof the size of the marker object image to the actual size of the markerobject. For example, in case that the marker object is a sheet of A4paper with a width of 210 mm and a length of 297 mm, the captured markerobject image has a width of 105 mm and a length of 148. 5 mm (that is,(marker object)/(marker object image)=2), and the selected product is a55 inch TV with a width of 1220 mm and a length of 690 mm, the productimage may have a width of 610 mm and a length of 345 mm.

Meanwhile, as illustrated in FIG. 3c , the electronic device 100 maydetermine a product corresponding to the handwriting image 310 as a TVproduct, and synthesize a TV image corresponding to the determined TVproduct with the space image. Here, the TV image may be synthesized in aregion in which the marker object is positioned.

FIGS. 4a to 4c illustrate another embodiment of the case that a markerobject is arranged on a floor.

As illustrated in FIG. 4a , the user may draw, on a marker object (forexample, a sheet of A4 paper), a shape (handwriting image 410) of aproduct that the user desires to arrange. Here, the handwriting image410 includes shape information and color information. For example, inFIG. 4a , the handwriting image 410 may be an image having a bluevertically long rectangular shape.

As illustrated in FIG. 4b , the user may position a marker object in aspecific region (for example, a floor) of a space (for example, a livingroom) in which a product is to be arranged. The electronic device 100may capture the space including the marker object to obtain an image ofthe space.

Similarly to the method as illustrated in FIGS. 3a to 3c , theelectronic device 100 may determine a product corresponding to thehandwriting image 410 as a furniture product 420, and synthesize afurniture image 420 with the space image as illustrated in FIG. 4c .Here, the furniture image may be synthesized in a region in which themarker object is positioned.

FIGS. 5a and 5b are illustrative views for describing a method ofchanging a product image included in an image with another productimage.

Specifically, as illustrated in FIG. 5a , the electronic device 100 mayrecognize a TV image 510 included in an image. Here, a method ofrecognizing a TV may be various. According to an embodiment, theelectronic device 100 may recognize a TV included in a space image byusing the fact that a screen size ratio of a general digital TV is 16:9.Specifically, the electronic device 100 may recognize an image having asize ratio of 16:9 among a plurality of product images included in thespace image, as the TV image 510. However, the disclosure is not limitedthereto, and the electronic device 100 may also recognize the TV byanalyzing color information included in the space image. For example, incase that the TV in the space image has a black screen, a rectangularimage including black color information may be recognized as a TV image.Further, even in case that a product included in the space image is nota TV, the electronic device 100 may obtain a product image by analyzingmain features of the corresponding product. For example, in case that aproduct included in the space image is a chair, the electronic device100 may analyze features such as chair legs, a chair backrest, and achair seat which are main components of the chair. Here, for productanalysis, the electronic device 100 may store information on variousproducts or may receive information on various products from an externalserver or the like.

As illustrated in FIG. 5b , in case that the electronic device 100recognizes the TV image 510 included in the space image and another TVproduct is selected by the user, the electronic device 100 maysynthesize a new TV image 520, instead of the existing TV image 510, ina region in which the existing TV image 510 is present. Here, in casethat information (size information, product information, and the like)on a TV corresponding to the existing TV image 510, and information on anew TV are input, the electronic device 100 may synthesize an accurateTV image 520 by using the above-described information. That is, in thiscase, the information on the existing TV may serve as a marker object.

Meanwhile, although the case that the marker object is not included inthe space image has been described in FIGS. 5a and 5b , it is a matterof course that the electronic device 100 may determine a size of theexisting TV image 510 and a size of the new TV image 520 by using animage including a marker object.

FIGS. 6a and 6b are illustrative views for describing a method ofdeleting a product image included in a space image according to anembodiment of the disclosure.

Specifically, as illustrated in FIG. 6a , the electronic device 100 mayrecognize one or more product images 610-1 and 610-2 included in a spaceimage. Here, in case that one 610-2 of the one or more product images isselected, the electronic device 100 may delete the selected productimage as illustrated in FIG. 6 b.

Here, a method of deleting the product image 610-2 may be various. Forexample, the electronic device 100 may delete the product image 610-2according to a command (for example, a command input by long pressing)input by touching a product image for a predetermined time.Alternatively, the electronic device 100 may delete the product image610-2 according to a command input by dragging, that is, by touching theproduct image 610-2 and moving the product image 610-2 to the outside ofthe space image. However, the disclosure is not limited to theembodiment described above, and the product image may be deleted byvarious methods. For example, the electronic device 100 may delete theproduct image by double tapping the product image included in the spaceimage, or may delete the product image through a UI (for example, aneraser UI) for deleting an image.

Meanwhile, as illustrated in FIG. 6b , the electronic device 100 mayperform image synthesis on a region 620 from which the product image610-2 is removed, based on an image pattern of a surrounding backgroundregion. Specifically, the electronic device 100 may analyze positioninformation of the deletion region 620 and image pattern information ofthe background region to obtain information on an image patternpositioned in the deletion region 620. Once the image patterninformation of the deletion region 620 is obtained, the electronicdevice 100 may perform image synthesis on the deletion region 620 basedon the obtained image pattern.

FIGS. 7a and 7b are illustrative views for describing a method ofrecommending a product according to an embodiment of the disclosure.

Specifically, the electronic device 100 may synthesize a product imageof a desired product with a space image by using the methods describedwith reference to FIGS. 3a to 6b . In this case, the electronicapparatus 100 may obtain information on other products related to thesynthesized product image, and recommend a product to the user.

Specifically, as illustrated in FIG. 7a , the electronic device maysynthesize a product image. Here, the synthesized product image may bean image of a display device capable of performing a playback ofcontents. The electronic device 100 may obtain a product related to thedisplay device based on product information corresponding to a displaydevice image 710, and recommend the product to the user.

Specifically, as illustrated in FIG. 7b , the electronic device 100 maydetermine that a speaker product matches the display device image 710,and synthesize speaker product images 720 and 730 corresponding to thespeaker product with the space image.

Here, the electronic device 100 may recommend a product that matches thedisplay device, based on a recommended product list and pre-definedrecommendation criteria. Specifically, the memory 140 of the electronicdevice 100 may store various product lists. Alternatively, theelectronic device 100 may receive a product list from an externalserver.

Meanwhile, a pre-defined recommendation list may vary depending on aproduct and a space where the product is applied. According to anembodiment, the electronic device 100 may be determined depending on aproduct installation type, an audio type, a product type, and the like.

For example, in case that the synthesized product is a digital TV, theelectronic device 100 may recommend a table for placing the digital TVbased on an installation type of the digital TV. That is, in case thatthe digital TV is a wall-mountable TV, the electronic device 100 may notrecommend a table, and in case that the digital TV is not awall-mountable TV, the electronic device 100 may recommend a table forplacing the digital TV. Alternatively, in case that the synthesizedproduct is a digital frame, the electronic device 100 may recommend awireless stereo system, a sound bar, or the like, that is connectable tothe digital frame, based on an audio type of the digital frame. Forexample, in case that the digital frame includes a component for audiooutput, the electronic device 100 may recommend a stereo system productcorresponding to the audio type of the digital frame. Alternatively, incase that the product to be synthesized is a digital frame, theelectronic device 100 may recommend various accessories related to thedigital frame, based on a product type. More detailed contents forproduct recommendation will be described below.

FIGS. 8a and 8b are illustrative views for describing another method ofsynthesizing a product image according to an embodiment of thedisclosure.

The embodiment described in FIGS. 3b to 7b relates to the case thatproduct information of a synthesized product image is stored in thememory 140. However, it is a matter of course that a product imageincluded in an image other than a space image as illustrated in FIG. 8amay be synthesized with the space image.

Specifically, as illustrated in FIG. 8a , the electronic device 100 mayextract a chair image 810 from an image including the chair image 810.As illustrated in FIG. 8b , the electronic device 100 may synthesize theextracted chair image 810 with a space image.

Here, in case that the image of FIG. 8a includes marker object or sizeinformation of the chair image 810 may be obtained, the electronicdevice 100 may synthesize a product image 810-1 that matches the spaceimage of FIG. 8b . In this case, the space image of FIG. 8b also needsto include a marker object or a product of which size information may beobtained. However, in case that the size information may not beobtained, the product image 810-1 may be changed by user operation.

FIGS. 9a to 9c are illustrative views for describing a method ofrepositioning a specific product image in a space image to anotherregion according to an embodiment of the disclosure.

Specifically, as illustrated in FIG. 9a , a space image may include alamp image 910. The electronic device 100 may obtain the lamp image 910by analyzing the space image.

As illustrated in FIGS. 9a and 9b , once user commands 900-1 and 900-2for repositioning the lamp image 910 are input, the electronic device100 may delete the lamp image 910 and change the lamp image 910 with alamp icon 920 corresponding to the lamp image 910.

Here, the user commands 900-1 and 900-2 may have various forms. Forexample, the user commands 900-1 and 900-2 may be a command input bylong pressing the lamp image 910, or may be various commands input bydouble tapping, swiping, dragging, or the like.

Once the lamp image 910 is deleted, the electronic device 100 mayperform image synthesis on a region from which the lamp image 910 isdeleted, based on an image pattern of a surrounding region. Meanwhile,the user command for repositioning the lamp image 910 may be a commandinput by touching the lamp image 910 for a predetermined time.

The electronic device 100 may edit (for example, moving, turning,resizing, or reversing the lamp image) the lamp icon 920 according tothe user command. That is, as illustrated in FIG. 9c , the electronicdevice 100 may re-synthesize the lamp image at a desired positionaccording to a user command.

FIGS. 10a to 10c are illustrative views for describing a method ofsynthesizing a virtual product image according to an embodiment of thedisclosure.

Specifically, as illustrated in FIG. 10a , the electronic device 100 mayinclude a list 1000-1 to 1000-8 including space images with which aproduct image is to be synthesized. Here, the list including spaceimages may include an image captured by a user and stored in the memory140, an image received from an external server, and the like. Further,the list may further include a UI element 1001 for capturing a spaceimage.

In case that a fifth space image 1000-5 is selected among the spaceimages included in the list, the electronic device 100 may display aproduct list 1010 to be synthesized with the selected space image, asillustrated in FIG. 10b . Here, the product list 1010 may be a productlist stored in the electronic device 10, or may be product list receivedfrom an external device. Further, it is a matter of course that theproduct list 1010 is a list of products selected by the user from a listincluding a plurality of products.

Further, the product list 1010 may display information (for example, aproduct name, a product type, a product size, and the like) on aproduct, in addition to an image of the product.

In case that a user command to select an electronic blackboard 1020 fromthe product list 1010 is input, the electronic device 100 may synthesizethe electronic blackboard 1020 with the space image. Here, a positionwhere the electronic blackboard 1020 is synthesized may be a positiondesignated by a user command. For example, in case that a user commandinput by dragging to a position desired by the user is input after theuser selects the electronic blackboard 1020, the electronic device 100may synthesize the electronic blackboard 1020 at the position desired bythe user.

However, the disclosure is not limited to the case that the userdesignates a position where the selected product is to be displayed, andit is a matter of course that the electronic device 100 may recommend aposition where the electronic blackboard 1020 is to be synthesized andsynthesize the electronic blackboard 1020 at the recommended position.Here, the electronic device 100 may display the recommended positionwhere the electronic blackboard 1020 is to be synthesized, in adistinguishable manner from that of other regions. Here, in case thatthe user selects the recommended position, the electronic device 100 maydisplay the electronic blackboard 1020 at the recommended position thatis selected.

As illustrated in FIG. 10c , an electronic blackboard image 1020-1synthesized according to the user command may be changed according tovarious embodiments described above. Meanwhile, in case that theelectronic blackboard image 1020-1 is synthesized, the electronic device100 may change the product list 1010 illustrated in FIG. 10b with arecommended product list 1030, and display the recommended product list1030. Here, the recommended product list 1030 may include products (forexample, the same product with a size different from that of theselected product, auxiliary products of the selected product, and thelike) related to the selected product, recommended products (forexample, other products that may be installed around the wall) that maybe installed at the selected position, and the like.

In case that synthesis of the product image is completed, the electronicdevice 100 may store an image obtained by the synthesis. Specifically,as illustrated in FIG. 10c , once a user command assigned to a save icon1040 is input, the electronic device 100 may store an image at a pointin time at which the user command is input.

Meanwhile, FIGS. 11a to 11d are illustrative views for describing amethod of arranging a new product in an image capable of expressing theproduct most fully.

Specifically, as illustrated in FIG. 11a , the electronic device 100 maydisplay UIs 1110-1, 1110-2, and 1110-3 classified into respectiveproduct categories. For example, the product categories may be dividedinto a SERIF TV UI 1110-1, a LIFE SQUARE UI 1110-2, and a PORTLAND UI1110-3. However, the disclosure is not limited to the categoriesdescribed above, and various category UIs may be further included.

In case that a user command to select the LIFE SQUARE UI 1110-1 isinput, the electronic device 100 may display a space image and a productlist UI 1120 corresponding to LIFE SQUARE as illustrated in FIG. 11b .In case that one product is selected from the product list UI 1120, theelectronic device 100 may synthesize an image of the selected productwith the space image and display the synthesized image. Here, once auser command assigned to the product list UI 1120 is input, theelectronic device 100 may change a size of the product and contentsdisplayed on the product.

Meanwhile, once a user command assigned to left and right switchingicons 1130 is input, the electronic device 100 may change and displaythe space image as illustrated in FIG. 11c . Here, the user command maybe a command input by touching the left and right switching UIs 1130, acommand input by swiping on the space image, or various other commands.

Further, once a user command input by touching the synthesized productimage is input, the electronic device 100 may display detailedinformation on the product as illustrated in FIG. 11d . Here, thedetailed information on the product may be at least one of a productadvertising phrase, a product price, a product size, or a manual.

FIG. 12 is a flowchart for describing a method for synthesizing avirtual product.

Specifically, the electronic device 100 may analyze a space image(S1210). In case that the space image includes a handwriting image(S1220-Y), the electronic device 100 may analyze shape information andcolor information of the handwriting image and obtain information on aproduct corresponding to the handwriting image in a pre-stored productlist (S1230).

Meanwhile, in case that the space image does not include a handwritingimage (S1220-N), the electronic device 100 may determine whether or notinformation on a product selected by a user command exists (S1240).

In case that the information on the product corresponding to thehandwriting image or the information on the product selected by the useris determined, the electronic device 100 may determine a position wherea product image corresponding to the product information is to besynthesized (S1250). Here, a region in which the product image is to besynthesized may be determined by a user command, or a region recommendedby the electronic device 100 through image analysis may be determined asthe region in which the product image is to be synthesized.

The electronic device 100 may determine whether or not another productimage is present in the region in which the product image is to besynthesized (S1260). In case that another product image is present inthe region in which the product image is to be synthesized (S1260-Y),the electronic device may delete the existing product image andsynthesize a new product image in the region from which the existingproduct image is deleted (S1270). Here, in case that a size of theexisting product image is larger than a size of the new product image,the electronic device 100 may perform image synthesis on a region otherthan a region in which the new product image is synthesized in anexisting product image region, based on an image pattern of asurrounding background region.

Meanwhile, in case that another product image is not present in theregion in which the product image is to be synthesized (S1260-N), theelectronic device may synthesize the new product image (S1280).

Hereinafter, a method of determining a size of a product image to besynthesized will be described in detail with reference to FIGS. 13 to16.

As described above, in case that a marker object is included in a spaceimage, the electronic device 100 may obtain size information of themarker object and size information of a marker object image. Therefore,the electronic device 100 may obtain size information of a product to besynthesized to calculate size information of a product image to besynthesized.

However, in case of a space image that does not include a marker object,the electronic device 100 may not obtain size information of a productimage to be synthesized. In this case, the electronic device 100 mayobtain information on a product corresponding to a product imageincluded in the space image to obtain size information of a productimage to be synthesized.

Specifically, the electronic device 100 may obtain a plurality ofproduct images by analyzing the space image. Here, in case that sizeinformation of an actual product corresponding to at least one productimage among the plurality of product images may be obtained, theelectronic device 100 may obtain a size of a product image to besynthesized. Here, the size information of the actual productcorresponding to at least one product image may be input by the user.However, the disclosure is not limited thereto, and it is a matter ofcourse that the electronic device 100 may determine a productcorresponding to a product image by analyzing the product image andobtain size information of the product based on information on thedetermined product.

Specifically, as illustrated in FIG. 13, the electronic device 100 mayobtain a first product image 1310 and a second product image 1320 from aspace image. The electronic device 100 may display the obtained productimages in a distinguishable manner. Here, in case that an actual width(for example, 50 cm) of the first product image is input, the electronicdevice 100 may display actual size information of the first productimage on the space image. Further, the electronic device 100 may obtaina size of a product image to be synthesized based on a size of the firstproduct image 1310 and the size information of a first product.

That is, as illustrated in FIG. 13, a product image 1330 to besynthesized may be obtained based on the actual size information of thefirst product, the size information of the first product image, andactual size information of a product to be synthesized.

Meanwhile, in case that the actual size information of the first productimage 1310 is input, the electronic device 100 may store a space imageincluding the actual size information of the first product image 1310 inthe memory 140. That is, the electronic device 100 may store a spaceimage and product information of a product image included in the spaceimage together, thereby making it possible to synthesize a product imagewithout analyzing the space image in case of arranging a virtual productby using the same space image in future.

Meanwhile, although a method of obtaining a size of a product image tobe synthesized based on a size of a product image has been described inFIG. 13, the disclosure is not limited thereto. For example, it is amatter of course that the electronic device 100 may synthesize a productimage at an accurate size also in case that a reference value such as alength of a wall included in a space image or a height of a space isinput.

FIGS. 14a and 14b are illustrative views for describing a method ofsynthesizing a product image by analyzing a perspective (or angle) of aspace image.

Specifically, even in case of an image of the same space, differentspace images are generated depending on a position and an angle at whichthe space is captured. Here, the electronic device 100 may synthesize anaccurate product image by analyzing a perspective of a space image.

For example, as illustrated in FIG. 14a , in case that a perspective ofa space image is not considered at the time of synthesizing a TV image1410 on a wall on the right side, the TV image 1410 is synthesized in aform different from that of an actually arranged TV.

Therefore, as illustrated in FIG. 14b , the electronic device 100 mayneed to arrange a TV image 1420 in the same form as that of the actuallyarranged TV by analyzing the perspective of the space image.

Here, the perspective of the space image may be obtained by usingvarious methods. For example, in case that the space image includes amarker object, the electronic device 100 may analyze the perspective ofthe space image based on a marker object image.

In case that the space image does not include a marker object, theelectronic device 100 may analyze the perspective of the space image byanalyzing a pattern of a wall, a boundary between a wall and a ceiling,and the like.

FIGS. 15a to 15c are illustrative views for describing the case ofzooming in or out a synthesized space image.

Specifically, as illustrated in FIG. 15a , a synthesized space image mayinclude a synthesized first product image 1510. Here, the electronicdevice 100 may zoom in or out the synthesized space image according to auser command. Here, the electronic device 100 may store the zoomed-in orzoomed-out space image and synthesize various product images byutilizing the stored space image.

As illustrated in FIG. 15b , once a user command to zoom in thesynthesized space image is input, the electronic device 100 may zoom inthe synthesized product image together with the synthesized space image.However, this is only an embodiment and it is a matter of course thatthe electronic device 100 may also zoom in or out only one of the spaceimage or the product image.

Meanwhile, as illustrated in FIG. 15c , the electronic device 100 maysynthesize a product image with the zoomed-in space image. Specifically,once a user command to change the first product image 1510 with a secondproduct image 1520 is input, the electronic device 100 may adjust a sizeof the second product image 1520 based on a magnification of thezoomed-in space image, and synthesize the second product image 1520.That is, even in case that the space image is zoomed-in or out, theproduct image may be synthesized in a form in which an actual product isarranged.

FIG. 16 is a flowchart for describing the embodiment of FIGS. 13 to 15.

First, the electronic device 100 may obtain a space image forsynthesizing a product image, and obtain a product image by analyzingthe space image (S1610).

The electronic device may determine whether or not product informationof the obtained product image is stored in the memory 140 (S1620).

In case that the product information of the obtained product image isnot input (S1620-N), the electronic device 100 may display one or moreproduct images obtained from the space image (S1630)

The electronic device 100 may obtain actual size information of one ofthe one or more product images (S1640). As described above, the actualsize information of the product image may be input by the user or may beobtained through image analysis. The electronic device 100 may store theobtained size information of the product.

Meanwhile, in case that product information of the obtained productimage is input (S1620-Y), the electronic device 100 may display the sizeinformation on the product image of which the size information is input(S1650).

The electronic device 100 may determine a product image to besynthesized and a region in which the product image is to be synthesized(S1660). The electronic device 100 may analyze a perspective of thedetermined region and synthesize a product image reflecting sizeinformation and perspective information in the determined region(S1670).

Once the product image is synthesized with the space image and a usercommand to zoom in or out the synthesized space image is input, theelectronic device 100 may zoom in or out the synthesized product imagetogether with the synthesized space image (S1680). Here, even in casethat another product image is synthesized, the product image may besynthesized with the space image by reflecting the size information andthe perspective information.

FIGS. 17 to 19 are illustrative views for describing a method ofrecommending an arrangeable product by analyzing a space image.

Specifically, the electronic device 100 may recommend a product to bearranged in a space image by analyzing color of the space image, aproduct image, and a density of product images. Further, in case that arecommended object is a display device such as a digital frame, theelectronic device 100 may also recommend a content to be played in thedisplay device. In addition, the electronic device 100 may also store arecommendation result to compile a database, and may recommend a similarproduct for a similar space image in future.

As illustrated in FIG. 17a , the electronic device 100 may analyze aspace image. For example, the electronic device 100 may analyze that thespace image is an image with blue colors, includes a chair image 1710,and includes a wall 1720 and a floor 1730 on which a product image maybe synthesized.

As illustrated in FIG. 17b , the electronic device 100 may recommend adigital frame 1740 and a region in which the digital frame 1740 is to bearranged, and synthesize a digital frame image. Further, as illustratedin FIG. 17c , the electronic device 100 may also recommend a content tobe displayed on a digital frame image 1740-1. Further, once a usercommand (for example, a command input by tapping a product image) forthe digital frame image 1740-1 is input, the electronic device 100 maychange a content displayed on the digital frame image 1740-1.

According to another embodiment, the electronic device 100 may furtherrecommend a product related to the recommended product. For example, asillustrated in FIG. 17d , in case that the electronic device 100recommends the digital frame 1740, the electronic device 100 may furtherrecommend an accessory such as a frame 1740-2 for the digital frame.

Meanwhile, the electronic device 100 may synthesize necessary additionalproducts together at the time of synthesizing a product with the spaceimage. Specifically, as illustrated in FIG. 18, in case that a TV image1810 is synthesized with a space image, the electronic device 100 maydisplay a position where a One Connect Box 8120, a set-top box 1830, orthe like connected to a TV product is to be installed.

Hereinafter, a process of recommending a product by analyzing a spaceimage will be described in more detail.

As described above, the electronic device 100 may obtain colorinformation, product image information, and density information byanalyzing a space image. Further, the electronic device 100 may obtaininformation on a space type, a furniture type, an electronic devicetype, and the like in the space image by analyzing the space image.Here, the space type may be information on whether a space correspondingto the space image is a living room, a restroom, or a bedroom. Thefurniture type may be information on a furniture image included in thespace image. The electronic device type may be information on varioustypes of electronic devices included in the space image.

The electronic device 100 may recommend a product based on a spaceanalysis result. Here, once a user command to select the recommendedproduct is input, the electronic device 100 may store space analysisdata, information on a finally selected product, and the like in thememory 140 to compile a database. However, it is a matter of course thatthe electronic device 100 may also store the information in an externalserver.

According to an embodiment of the disclosure, the electronic device 100may tabulate and store a space ID, space analysis data, a productselected by the user, and a space type.

TABLE 1 Finally Selected Product (Number Space Space ID Data of Times)Type Space1 Place: Living Room Sound Bar: 5 Type A Presence or Absenceof Table: 2 Device: Digital TV Frame: 1 Density in Space: Low . . .Color Distribution: White Furniture Type: Sofa Space2 Place: Kitchen TV:0 Type B Presence or Absence of Frame: 4 Device: Speaker Device Chair: 6Density in Space: Medium . . . Color Distribution: Brown Furniture Type:Table . . . . . . . . . . . .

In case that a database as Table 1 is complied, the electronic device100 may recommend a product based on a space analysis result and thedatabase for the space image. Specifically, in case that a space imagesimilar to a space image analysis result exists in the database, theelectronic device 100 may recommend a recommended product stored in thedatabase.

For example, the electronic device 100 may recommend a sound bar in casethat the analyzed space image is similar to the space type of Type A,and the electronic device 100 may recommend a chair in case that theanalyzed space image is similar to the space type of Type B.

Meanwhile, in case that the recommended product is a display device, theelectronic device 100 may recommend a content to be provided to thedisplay device. Here, the electronic device 100 may recommend a content,based on a category (product type, space type, or the like) andcategory-specific detailed criteria.

TABLE 2 Category Detailed Criteria Recommended Contents Product ProductSpecialized Art Image or Video that Matches Type for Interior SpaceColor Decoration Product Specialized Video Contents for Watching MediaProduct Proportion Media Specialized for Proportion of Product ProductSize Large-sized: Video Contents Small- and Medium-sized: SoundVisualizer, Work of Art, and the like Space Living Room Video ContentsType Kitchen and Bedroom Work or Art, Clock, and the like

As shown in Table 2, the electronic device 100 may recommend a suitablecontent based on a type of recommended display device and a type ofspace in which the display device is to be installed.

Here, a deep-learning-based artificial intelligence technology may beused to recommend a product or contents to the user based on spaceanalysis and database. The deep-learning-based recommendation methoddescribed above may be performed in the electronic device 100. However,it is a matter of course that the deep-learning-based recommendationmethod may be performed in an external server and a result thereof maybe received by the electronic device 100.

For example, in case that the deep-learning-based recommendation isperformed in an external server, the external server may recommend aproduct based on a data recognition model created by using a learningalgorithm. Here, the external server may include a data learner and adata recognizer.

Here, the data learner may create a data recognition model havingcriteria for determining a certain situation or may make the datarecognition model to learn the criteria for determining a certainsituation. In order to determine a certain situation, the data learnermay apply learning data to a data recognition model to create a datarecognition model having determination criteria.

The data recognizer may determine a situation based on recognition data.The data recognizer may determine a situation based on predeterminedrecognition data by using the learned data recognition model. The datarecognizer may obtain predetermined recognition data according topredetermined criteria, and apply the obtained recognition data, as aninput value, to the data recognition model, to thereby determine (orestimate) a certain situation based on the predetermined recognitiondata.

Further, a result value output by applying an input value for theobtained recognition data to the data recognition model may be used toupdate the data recognition model.

Meanwhile, the data learner may include a data obtainer, apre-processor, a learning data selector, a model learner, and a modelevaluator, and the data recognizer may include a data obtainer, apre-processor, a recognition data selector, a recognition resultprovider, and a model updater.

FIG. 19 is a flowchart for describing a process of synthesizing aproduct image according to an embodiment of the disclosure.

First, the electronic device 100 may capture a marker object including ahandwriting image to obtain a space image (S1910). As described above,it is possible to obtain a size of a product image with respect to anactual size of a product based on a ratio of the size of a marker objectimage to a size of the marker object.

The electronic device 100 may obtain information on one or more productscorresponding to the handwriting image included in the marker object(S1920). The handwriting image may be a drawing drawn by the user, andmay include at least one of color information or shape information.

Once one of the obtained one or more products is selected through theinputter 130, the electronic device 100 may synthesize and display aproduct image in and on a region in which a marker object image ispositioned, based on information on the selected product, sizeinformation of the marker object, and size information of the markerobject image (S1930).

In addition, the control method of an electronic device according to thevarious embodiments described above may be implemented by a program tothereby be provided to the display device. Particularly, a problemincluding the display method may be stored and provided in anon-transitory computer readable medium.

The non-transitory computer readable medium is not a medium that storesdata therein for a while, such as a register, a cache, or a memory, butmeans a medium that semi-permanently stores data therein and is readableby a device. In detail, the various applications or programs describedabove may be stored and provided in the non-transitory computer readablemedium such as a compact disk (CD), a digital versatile disk (DVD), ahard disk, a Blu-ray disk, a universal serial bus (USB), a memory card,a read only memory (ROM), or the like.

Although the embodiments of the disclosure have been illustrated anddescribed hereinabove, the disclosure is not limited to theabovementioned specific embodiments, but may be variously modified bythose skilled in the art to which the disclosure pertains withoutdeparting from the scope and spirit of the disclosure as disclosed inthe accompanying claims. These modifications should also be understoodto fall within the scope of the disclosure.

What is claimed is:
 1. A control method of an electronic deviceincluding a camera and a display, the control method comprising:obtaining, by a camera, a captured image including a marker object onwhich a sketch is drawn; obtaining information on a plurality ofproducts corresponding to the sketch by analyzing the captured image;displaying the information on the plurality of products on the display;based on a first user command for selecting a first product among theplurality of products, obtaining a first image including the firstproduct in a region in which the marker object is positioned; displayingthe first image on the display; based on a second user command forselecting a second product among the plurality of products, obtaining asecond image including the second product in a region in which the firstproduct is positioned; and displaying the second image on the display.2. The control method as claimed in claim 1, wherein in the obtainingthe first image, the first image is processed by analyzing sizeinformation of the first product, size information of the marker object,size information of the marker object image, and perspective informationof the marker object image.
 3. The control method as claimed in claim 1,wherein the captured image includes color information, and in theobtaining of the information the plurality of products, information onthe plurality of products corresponding to the sketch and colorinformation of the captured image is obtained.
 4. The control method asclaimed in claim 1, further comprising: based on a user command todelete one of a plurality of product images included in the capturedimage is input, deleting the one product image; and performing imagesynthesis on a region from which the product image is deleted based onan image pattern of a surrounding region of the deleted product imageand displaying a synthesized image.
 5. The control method as claimed inclaim 1, further comprising: based on one of a plurality of productimages included in the captured image being selected, obtaining productinformation of the selected product image; and recommending new productinformation based on the obtained product information.
 6. The controlmethod as claimed in claim 1, further comprising: obtaining colorinformation of the captured image; and recommending new productinformation based on a plurality of product information included in thecaptured image and the color information.
 7. The control method asclaimed in claim 1, wherein in the displaying, based on a user commandfor specifying one of a plurality of product images included in thecaptured image being input, the one product image specified in the usercommand is repositioned and displayed based on the user command.
 8. Thecontrol method as claimed in claim 1, wherein the information on theplurality of products include at least one of size information, categoryinformation, or price information of a product.
 9. An electronic devicecomprising: a camera configured to capture an image; a displayconfigured to display the image; an inputter configured to receive auser command; at least one processor; and a memory configured to storeone or more computer programs executed by the at least one processor,wherein the one or more computer programs include: a command to controlthe camera to capture a marker object on which a sketch is drawn toobtain a captured image; a command to obtain information on a pluralityof products corresponding to the sketch by analyzing the captured image;a command to control the display to display the information on theplurality of products; a command to obtain a first image including afirst product in a region in which the marker object is positioned basedon a first user command for selecting the first product among theplurality of products; a command to control the display to display thefirst image; a command to obtain a second image including a secondproduct in a region in which the first product is positioned based on asecond user command for selecting the second product among the pluralityof products; and control the display to display the second image. 10.The electronic device as claimed in claim 9, wherein the command toobtain the first image includes a command to synthesize and process thefirst image by analyzing size information of the first product, sizeinformation of the marker object, size information of the marker objectimage, and perspective information of the marker object image.
 11. Theelectronic device as claimed in claim 9, wherein the captured imageincludes color information, and the command to obtain the information onthe plurality of products includes a command to obtain information onthe plurality of products corresponding to the sketch and the colorinformation of the captured image.
 12. The electronic device as claimedin claim 9, wherein the one or more computer programs further include: acommand to receive a user command to delete one of a plurality ofproduct images included in the captured image, a command to delete theone product image; and a command to perform image synthesis andprocessing on a region from which the product image is deleted based onan image pattern of a surrounding region of the deleted product image.13. The electronic device as claimed in claim 9, wherein the one or morecomputer programs further include: a command to obtain, based on one ofa plurality of product images included in the captured image isselected, product information of the selected product image; and acommand to recommend new product information based on the obtainedproduct information.